In relation to image processing, various related technologies have been known.
For example, as an example of a technology to generate a restored image (for example, a high-resolution image) from an input image (for example, a low-resolution image), super-resolution technology is known. Among the super-resolution technologies, a technology to generate a high-resolution image using a dictionary is, in particular, referred to as a study based super-resolution technology. The dictionary mentioned above is a dictionary that are created through studying cases each of which includes a study image (in general, a high-quality image) and a degraded image corresponding to the study image (for example, an image created by reducing the resolution of the study image). The restored image generated by the super-resolution technology is also referred to as a super-resolution image.
PTL 1 discloses an example of a character recognition device. The character recognition device disclosed in PTL 1 performs super-resolution processing to recognize characters on a license plate or the like, which are included in an object image taken with a camera.
The character recognition device performs the super-resolution processing by using a database (dictionary) in which low-resolution dictionary images, feature values of the low-resolution dictionary images, and high-resolution dictionary images are associated with one another. The low-resolution dictionary images mentioned above are images of characters that have been taken with the camera with which the object image is taken. The feature values are feature values that are calculated on the basis of respective ones of the low-resolution dictionary images. The high-resolution dictionary images are images of characters that have been taken with a camera that has a higher resolution compared with the camera with which the object image is taken.
PTL 2 discloses an example of a super-resolution image processing device. The super-resolution image processing device disclosed in PTL 2 outputs a high-resolution image from a low-resolution original image (input image data).
The super-resolution image processing device uses a dictionary table and others, which have been generated by a dictionary creation device, to infer lost high frequency components in generating output image data through applying super-resolution image processing to the input image data. The dictionary table and others mentioned above are a dictionary table, a first principal component basis vector, and a second principal component basis vector. The dictionary creation device generates the dictionary table and others that are optimized for a specific scene by the following procedure.
First, the dictionary creation device acquires a sectioned bitmap, which is a processing object, from a sample image file, breaks down the acquired bitmap into a plurality of broken bitmaps, and stores the broken bitmaps in records in a temporary table.
Next, the dictionary creation device applies MP (Max-Plus) wavelet transformation processing, permutation processing, principal component analysis processing, inner product operation processing, and frequency partition processing to the broken bitmaps in order, and stores results of the processing in respective fields in the temporary table. In the principal component analysis processing, the dictionary creation device calculates the first principal component basis vector and the second principal component basis vector.
Last, the dictionary creation device creates the dictionary table, which has a smaller number of records compared with the temporary table, using a mean value operation unit. The dictionary table differs from the dictionary of the above-described study based super-resolution technology. That is, the dictionary table is not a dictionary that is created through studying cases in which study images are associated with degraded images.
PTL 3 discloses an example of an image super-resolution device. The image super-resolution device disclosed in PTL 3 generates a super-resolution image that is enlarged with a preset enlargement ratio from an input image degraded due to encoding and decoding. The encoding and decoding mentioned above are encoding and decoding by a preset encoding method. Specifically, the image super-resolution device generates a super-resolution image through the following processing.
First, the image super-resolution device partitions an input image into blocks of a prefixed size, and cuts out respective ones of the blocks as processing blocks. Next, the image super-resolution device generates enlarged processing blocks by enlarging the processing blocks with a prefixed enlargement ratio. The prefixed enlargement ratio is an enlargement ratio with which the image super-resolution device enlarges the input image when the image super-resolution device generates the super-resolution image.
Second, the image super-resolution device writes reference blocks and degraded reference blocks in association with each other in a block storage means. The reference blocks mentioned above are blocks that are cut out from the input image and have the same size as that of the processing blocks. The degraded reference blocks mentioned above are blocks into which the reference blocks are degraded by a specific degradation process. The specific degradation process is a degradation process when it is assumed that the input image is an image into which the to-be-generated super-resolution image has been degraded through the degradation process. Specifically, the image super-resolution device degrades the reference blocks using a degradation model based on an encoding method by which the input image has been degraded (a model that simulates predefined orthogonal transformation, quantization, and so on) to generate the degraded reference blocks.
Third, the image super-resolution device calculates similarities between the degraded reference blocks and the processing blocks.
Fourth, the image super-resolution device enlarges the degraded reference blocks with the prefixed enlargement ratio to generate restored reference blocks. Further, the image super-resolution device calculates differences between the restored reference blocks and the reference blocks as loss components.
Fifth, the image super-resolution device combines the enlarged processing blocks with the loss components on the basis of the similarities to generate super-resolution blocks. The image super-resolution device constructs the super-resolution blocks into an image to generate the super-resolution image into which the input image is enlarged.